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TITLE OF THE INVENTION 

Image Processing Apparatus 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present invention relates to an image processing apparatus. 

Description of the Background Art 

In response to a recent trend towards higher pixel resolution of a digital camera, 
10 higher transmission speed and higher image processing speed of captured image data 
have been required. Figs. 7 and 8 each schematically show a flow of image data 
captured by a conventional digital camera. An interlace sensor 2 is used as a CCD 
sensor for image capturing in Fig. 7, whereas a progressive sensor 18 is used in Fig. 8. 
With reference to Figs. 7 and 8, image processing operation of a conventional digital 
1 5 camera will be discussed below. 

With reference to Fig. 7, when a CCD sensor for image capturing is the 
interlace sensor 2, the interlace sensor 2 first converts captured information as light about 
a subject into a digital signal which is then outputted as image data. The outputted 
digital signal is subjected to noise control by correlated dual sampling, A/D conversion 
20 into a digital signal and the like at a signal processing unit 3, and is thereafter sent to a 
main processor 4. The main processor 4 comprises a sensor processing unit (hereinafter 
referred to as “SPU”) 6 for performing signal processing such as white balance control on 
the received image data. Next, the processed image data is stored in a main memory 5. 

The interlace sensor 2 outputs the captured image of one frame in two fields, 
25 namely, odd and even fields. However, image processing using the SPU 6 and later 
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units requires peripheral pixel information, and hence, information only about the odd 
field or about the even field is not responsive to the image processing. In response, 
subsequent process steps are put on hold until the interlace sensor 2 outputs the image 
data of two fields to reproduce the image data of one frame as original image data 16 on 
5 the main memory 5 . 

After the image data of one frame is reproduced as the original image data 16, a 
real-time processing unit (hereinafter referred to as “RPU”) 7 performs signal processing 
such as interpolation, color space conversion and pseudo color suppression on the image 
data written to a line memory 8 from the main memory 5. The image data processed by 
10 the RPU 7 is thereafter sent to an image compression and expansion unit 13 through line 
buffers A24 and B25 on the main memory 5. For compression into the JPEG (Joint 
Photographic Experts Group) format, the line buffers A and B each normally have a 
capacity of 8 lines. 

More specifically, after the processed image data is sent to the line buffer A24, 
1 5 the RPU 7 continues to process subsequent image data read from the original image data 
16, which processed data is then sent to the line buffer B25. The image data sent in 
advance to the line buffer A24 is concurrently sent to the image compression and 
expansion unit 13 and is subjected to processing therein. The RPU 7 continues to be 
operative to send subsequent image data to the empty line buffer A24, concurrently with 
20 which the image data sent in advance to the line buffer B25 is sent to the image 
compression and expansion unit 13. That is, the two line buffers A24 and B25 become 
alternately operative. 

The image data read from the line buffers A24 and B25 are subjected to 
compression into a predetermined format such as JPEG at the image compression and 
25 expansion unit 13, eind are thereafter sent as compressed image data 17 to the main 
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memory 5. The compressed image data 17 is subjected to processing by an image 
display unit 19 and a signal conversion unit 20, whereby an image is displayed on a 
display device 2 1 , or the compressed image data 1 7 is stored in a memory card through a 
memory card interface. The image display unit 1 9 uses a display buffer 29 as a working 
5 buffer memory. 

Unlike the interlace sensor 2, the progressive sensor 18 as a CCD sensor for 
image capturing outputs the captured image of one frame at a time, and hence, generation 
of the original image data 16 on the main memory 5 is not required. Except for this, 
image processing using the progressive sensor 1 8 follows the same flow as the foregoing 
10 image processing using the interlace sensor 2. 

As a result of higher pixel resolution, the CCD sensors 2 and 1 8 each have a 
growing number of pixels per horizontal line, reaching such a length that exceeds the 
horizontal line length of the line memory 8 of the RPU 7 in some cases. On the 
occurrence of such excess, regardless of whether the CCD sensor 2 or 18 is used, the 
15 image data outputted from the CCD sensor 2 or 18 cannot be directly processed by the 
RPU 7. In response, even in the use of the progressive sensor 18, the original image 
data 16 is generated once on the main memory 5, and thereafter the subsequent process 
steps are followed. More specifically, with reference to Fig. 9, the original image data 
1 6 is divided into left and right regions to avoid excess of the pixels per horizontal line of 
20 the original image data 1 6 over the horizontal line length of the line memory 8 of the RPU 
7. The left region including regional data P| through P(n/ 2 ) is processed first to generate a 
region 27 including regional data Qi through Q(n/2>- Next, the right region including 
regional data P(n/2)+ i through P„ is processed to generate a region 28 including regional 
data Q(n/2) + i through Q„. Finally, two regional data Ri and R2 on the left and right are 
25 synthesized to form intermediate data 26 of one frame on the main memory 5 which is 
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then processed by the image compression and expansion unit 13. 

As discussed, for compression of a subject image captured by a digital camera 
into a predetermined format such as JPEG to be displayed on a display device or stored in 
a memory card, image data transmission should be repeated between the main processor 4 
5 and the main memory 5 through a bus 14. More specifically, a digital camera using the 
interlace sensor 2 requires five transmissions through the bus 14 as shown in Fig. 7. 
Even a digital camera using the progressive sensor 1 8 requires three transmissions. 

As discussed, the CCD sensors 2 and 18 have a horizontal line of pixels 
reaching a length that exceeds the horizontal line length of the line memory 8 of the RPU 
10 7, in which case the regional data R| and R 2 on the left and right separately processed are 

synthesized into the intermediate data 26 which is then held in the main memory 5. Asa 
result, the main memory 5 should be larger in capacity than required in the case in which 
the CCD sensors 2 and 18 contain pixels per horizontal line which do not reach the 
horizontal line length of the line memory 8 of the RPU 7. 

15 The foregoing data transmissions are handled by a DMA control unit 15 which 

realizes high-speed DMA transmission through the bus 14 with no intervention of a CPU. 
However, when the CCD sensors 2 and 1 8 are a 5-megapixel-class CCD sensor with 2560 
pixels high by 1920 pixels wide each storing 16 bits of information, for example, the 
image data outputted from the CCD sensor 2 or 18 contains information as much as about 
20 10 megabytes. In light of a recent trend towards still higher pixel resolution of the CCD 

sensors 2 and 18, repetition of transmission of such high- volume data causes heavily 
places a burden on a bus band, thus leading to increase in power consumption, lower 
processing speed, and the like. 
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SUMMARY OF THE INVENTION 
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The present invention is intended for an image processing apparatus in which 
an image processing unit writes image data to a buffer memory, and a compression unit 
reads the image data from the buffer memory to perform compression thereon. 

According to the present invention, the image processing apparatus comprises: 
5 an image processing part for performing image processing on captured data of an image; 
and a storage unit provided outside the image processing part. The image processing 
part includes: an image processing unit for performing a predetermined process on the 
captured data to obtain image data, and outputting the image data; a buffer memory for 
storing the image data sent from the image processing unit; and a compression unit for 
10 compressing the image data read from the buffer memory. 

Accordingly, high-speed processing of the image data is realized without using 
the storage unit. Reduction in power consumption is allowed as well. 

According to one aspect of the present invention, the buffer memory includes 
two buffer memories. The image processing apparatus further comprises a control unit 
15 for controlling read and writing to and from these two buffer memories. The control 
unit is operative in such a manner that while the image processing unit writes image data 
to either one of the two buffer memories, the compression unit selectively reads image 
data previously stored in the other one of the two buffer memories. As a result, the 
image processing unit and the compression unit are both allowed to perform sequential 
20 image processing without the need for waiting for completion of data transmission to the 
buffer memories. 

It is therefore an object of the present invention to provide an image processing 
apparatus realizing high-speed image processing with low power consumption. 

These and other objects, features, aspects and advantages of the present 
25 invention will become more apparent from the following detailed description of the 
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present invention when taken in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 schematically shows the configuration of a digital camera using an 
5 interlace sensor according to a first preferred embodiment of the present invention; 

Fig. 2 schematically shows the configuration of a digital camera using a 
progressive sensor according to the first preferred embodiment; 

Fig. 3 schematically shows the configuration of a digital camera according to a 
second preferred embodiment of the present invention; 

10 Fig. 4 schematically shows the configuration of a digital camera according to a 

third preferred embodiment of the present invention; 

Fig. 5 schematically shows the configuration of a digital camera according to a 
fourth preferred embodiment of the present invention; 

Fig. 6 shows process flow of regional data according to the fourth preferred 
1 5 embodiment; 

Fig. 7 schematically shows the configuration of a conventional digital camera 
using an interlace sensor; 

Fig. 8 schematically shows the configuration of a conventional digital camera 
using a progressive sensor; and 

20 Fig. 9 shows conventional process flow of regional data. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
First Preferred Embodiment 

Fig. 1 schematically shows a digital camera comprising an image processing 
25 apparatus 1 according to a first preferred embodiment of the present invention. In Fig. 1, 
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arrows indicate flow of image data captured by the digital camera. With reference to Fig. 
1, the image processing apparatus 1 comprises a main processor 4 and a main memory 5. 
The image processing apparatus 1 is cormected through a signal processing unit 3 to an 
interlace sensor 2. 

5 The main processor 4 comprises an SPU 6, an RPU 7 including a line memory 

8, an image compression and expansion unit 13, a local buffer A9 (first buffer memory) 
and a local buffer BIO (second buffer memory) serving to treinsmit data between the RPU 
7 and the image compression and expansion unit 13, a write control unit 11 and a read 
control unit 12 for selecting which one of the two local buffers A9 or BIO to use, a bus 14 
10 for data transmission, and a DMA control unit 15 for controlling data transmission 
through the bus 14 between the main processor 4 and the main memory 5. 

The main memory 5 serves to temporarily store original image data 16 
generated in the main processor 4 by data processing of odd and even fields sent from the 
interlace sensor 2 through the signal processing unit 3, and compressed image data 17 
15 sent from the image compression and expansion unit 13. 

Except for the two local buffers A9 and BIO provided between the RPU 7 and 
the image compression and expansion unit 13, and operation of the main memory 5 as a 
line buffer, the digital camera of the first preferred embodiment has the same 
configuration as that of the conventional digital camera. 

20 Though not specifically shown, the main processor 4 further comprises a CPU 

for various types of operations, an external memory card for storing captured image data, 
hardware serving as an interface for storage into the external memory card, and a storage 
device for storing a software program. 

The operation of the image processing apparatus 1 having the foregoing 
25 configuration will be discussed below. 
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In the main processor 4, the SPU 6 receives image data from the interlace 
sensor 2 through the signal processing unit 3 to generate the original image data 1 6 on the 
main memory 5. Next, the RPU 7 stores image data in the line memory 8 read from the 
original image data 1 6 and performs signal processing thereon such as interpolation, color 
5 space conversion and pseudo color suppression. These process steps are the same as 
those followed by the conventional digital camera discussed with reference to Fig. 7. 
Unlike the conventional digital camera, the digital camera of the first preferred 
embodiment sends the processed image data to the local buffers A9 and BIO in the main 
processor 4, not to the line buffers A24 and B25 in the main memory 5. 

10 More specifically, the processed image data of 8 lines is sent to the local buffer 

A9. The RPU 7 thereafter continues to process subsequent image data of 8 lines read 
from the original image data 16, and sends the processed image data to the local buffer 
BIO. Concurrent with transmission of the image data from the RPU 7 to the local buffer 
BIO, the image data stored in the local buffer A9 is sent to the image compression and 
15 expansion unit 13 for compression into a predetermined format such as JPEG. The 

compressed image data is thereafter sent to the main memory 5. The RPU 7 still 

continues to process subsequent image data of 8 lines read from the original image data 
16 and send the processed image data to the empty line buffer A9. The image data 
stored in the local buffer BIO is concurrently sent to the image compression and 

20 expansion unit 13 for compression into a predetermined format such as JPEG. The 

compressed image data is thereafter sent to the main memory 5. That is, while the RPU 
7 sends image data either to the local buffer A9 or to BIO, image data stored either in the 
local buffer BIO or in A9 is sent to the image compression and expansion unit 13 to be 
compressed therein. These operations are repeatedly performed, whereby the 
25 compressed image data 1 7 is generated on the main memory 5. 
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Fig. 2 shows the configuration of a digital camera using a progressive sensor 18 
as a CCD sensor. Except for the progressive sensor 1 8, the digital camera shown in Fig. 
2 has the same configuration as that of the digital camera of Fig. 1. In Fig. 2, constituent 
elements which are operative in the same manner as those of Fig. 1 are designated by the 
5 same reference numerals. The operation of the digital camera using the progressive 

sensor 18 is the same as that of the digital camera using the interlace sensor 2, with 
exception that no original image data 1 6 is required, and data from the SPU 6 is directly 
sent to the RPU 7 to be processed therein. 

The write and read control units 11 and 12 select which one of the two local 
10 buffers A9 or BIO to use by making connection to the RPU 7 and the image compression 
and expansion unit 13. 

The line buffers A9 and BIO are only required to be not less than image data 
processed at one time by the RPU 7 in horizontal line length. More specifically, as long 
as the local buffers A9 and BIO have a horizontal line not less than one horizontal line of 
15 pixels of the CCD sensors 2 and 18, the local buffers A9 and BIO may be the same as or 

shorter than in horizontal line length the line memory 8 of the RPU 7. By way of 

example, when the CCD sensors 2 and 18 are a 5-megapixel-class CCD sensor with 2560 
horizontal pixels, and the line memory 8 of the RPU 7 has a horizontal line length with 
4096 pixels, the horizontal line length of the local buffers A9 and BIO may contain 2600 
20 pixels or 4096 pixels, as long as it is not less than the length of 2560 pixels. 

As discussed, the use of the local buffers A9 and BIO reduces the number of 
DMA transmissions of image data between the main processor 4 and the main memory 5 
under control by the DMA control unit 15 until the compressed image data 17 is 
generated. That is, the conventional digital camera requires five transmissions in the use 
of the interlace sensor 2 and three transmissions in the use of the progressive sensor 1 8 as 
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shown in Figs. 7 and 8, respectively, whereas with reference to Figs. 1 and 2, the digital 
camera of the first preferred embodiment only requires three transmissions and one 
transmission in the use of the interlace sensor 2 and the progressive sensor 18, 
respectively. A burden placed on a bus band is lightened accordingly, whereby 
5 high-speed image processing is realized. The first preferred embodiment further 
advantageously serves to reduce the frequency of use of the main memory 5 generally 
operating at a higher voltage than the main processor 4 as well as to reduce the number of 
operations of the DMA control unit 15, thereby reducing power consumption of the image 
processing apparatus 1 as a whole. Additionally, provision of two local buffers allows 
10 continuous process flow from the SPU 6 to the image compression and expansion unit 13, 
thereby advantageously realizing high-speed image processing. 

Second Preferred Embodiment 

Under the conditions that the progressive sensor 18 is used as a CCD sensor, 
and the progressive sensor 1 8 has a horizontal line of pixels that does not exceed in length 
15 the horizontal line of the local buffers A9 and BIO, the first preferred embodiment allows 
real-time data processing flow from the SPU 6 to the image compression and expansion 
unit 13 by way of the local buffers A9 and BIO without generating the original image data 
16 once on the main memory 5 as shown in Fig. 2. When the CCD sensors 2 and 18 
have a horizontal line of pixels that exceeds in length the horizontal line length of the 
20 local buffers A9 and BIO, however, data sent from the RPU 7 cannot completely be 
stored in the local buffers A9 and BIO, whereby such sequential processing becomes 
impossible. 

In a second preferred embodiment of the present invention, it will be discussed 
what process a digital camera follows to handle the case in which the CCD sensors 2 and 
25 18 have a horizontal line of pixels exceeding in length the horizontal line length of the 
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local buffers A9 and BIO. 

Fig. 3 schematically shows a digital camera comprising an image processing 
apparatus 1 according to the second preferred embodiment. In Fig. 3, arrows indicate 
flow of image data captured by the digital camera. The image processing apparatus 1 of 
5 the second preferred embodiment has the same configuration as that of the first preferred 
embodiment, whereas the local buffers A9 and BIO are operative in different ways 
between the first and second preferred embodiments. In Fig. 3, constituent elements 
which are operative in the same manner as those of Figs. 1 and 2 are designated by the 
same reference numerals. 

10 As an example, assuming that the CCD sensors 2 and 18 are a 

6-megapixel-class CCD sensor with 3072 pixels per horizontal line, that the line memory 
8 of the RPU 7 contains 4096 pixels per horizontal line, and that the local buffers A9 and 
BIO each have 2600 pixels per horizontal line responsive only to a 5-megapixel-class 
CCD with 2560 pixels per horizontal line, the local buffers A9 and BIO are overflowed 

1 5 with data sent from the RPU 7 which prevents process flow. In response, in the second 
preferred embodiment, the two local buffers A9 and BIO together serve as one continuous 
buffer memory as shown in Fig. 3. As a result, the horizontal line length of the local 
buffers A9 and BIO becomes responsive to data up to 5200 pixels per horizontal line 
which is twice the horizontal line length of each one of the local buffers A9 and BIO. 

20 The use of the local buffers A9 and BIO as one continuous buffer memory does 

not allow the RPU 7 and the image compression and expansion unit 13 to follow 
sequential process flow by means of the alternate use of the two local buffers A9 and BIO 
as discussed in the first preferred embodiment. In response, a control signal is 
transmitted between the RPU 7 and the image compression and expansion unit 1 3 . 

25 More specifically, the RPU 7 first processes image data of one line read fi-om 




12 



the original image data 1 6, and sends the processed image data to the buffer memory as a 
unity of the two local buffers A9 and BIO. At the time when data transmission is 
completed, the RPU 7 stops the operation. The image compression and expansion unit 
13 serves to process the data read from the buffer memory. After receipt of all the data 
5 in the buffer memory, the image compression and expansion unit 13 sends a control 
signal to the RPU 7 notifying completion of data readout. On receipt of this control 
signal, the RPU 7 again processes image data read from the original image data 16, and 
sends the processed image data to the buffer memory. 

The second preferred embodiment is applicable even in the use of the 
10 progressive sensor 18 as a CCD sensor. More specifically, data from the SPU 6 is not 
directly sent to the RPU 7, but is sent to the main memory 5 once to generate the original 
image data 16. 

In Fig. 3, the two local buffers A9 and BIO are conceptually shown as one 
buffer memory in physical connection in a horizontal direction. The actual use of the 
1 5 local buffers A9 and B 1 0 is such that under control by a software program, for example, 

the RPU 7 sends data to the local buffer A9, and overflow data from the local buffer A9 is 
sequentially sent to the local buffer B 1 0. This process flow is also followed in reading 
to the image compression and expansion unit 13. When the write and read control units 
11 and 12 are collectively controlled, the second preferred embodiment is feasible in the 
20 configurations shown in Figs. 1 and 2. That is, the configurations of Figs. 1 and 2 can 
be subjected to the process flows of the first and second preferred embodiments. 

The second preferred embodiment has been described in the use of the two local 
buffers A9 and BIO as discussed in the first preferred embodiment. As the simplest way 
to use a buffer memory, one local buffer may be provided in the image processing 
apparatus 1 that has the same capacity as that of the line memory 8 of the RPU 7. The 
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second preferred embodiment is also feasible by the use of such a local buffer and the 
foregoing control signal. 

As discussed, even when the CCD sensors 2 and 18 have high pixel resolution 
with a horizontal line of pixels exceeding in length the horizontal line length of the two 
5 local buffers A9 and BIO, the foregoing use of one buffer memory as a unity of the local 
buffers A9 and BIO realizes image processing. Even when only one buffer memory is 
prepared which has a horizontal line length exceeding the length of one horizontal line of 
pixels of the CCD sensors 2 and 18, the use of the foregoing control signal realizes image 
processing. Like the first preferred embodiment, data transmission between the RPU 7 
10 and the image compression and expansion unit 13 does not require the main memory 5 
and the bus 14. A burden placed on a bus band is lightened accordingly, whereby 
high-speed image processing and reduction in power consumption are realized. 

Third Preferred Embodiment 

In the first and second preferred embodiments, the local buffers A9 and BIO 
15 serve to transmit data from the RPU 7 to the image compression and expansion unit 13. 
The local buffers A9 and BIO may be operative for another processing performed in the 
main processor 4. In a third preferred embodiment of the present invention, the local 
buffers A9 and BIO are operative for image display on a display device 21. 

For visual recognition of a subject to be captured, a digital camera may use an 
20 electronic viewfinder or a liquid crystal display. With reference to Figs. 7 and 8, in the 
conventional way of such visual recognition, image data sent from the CCD sensor 2 or 
18 passes through the signal processing unit 3 and the SPU 6, and is thereafter sent 
through the bus 14 to the image display unit 19 for conversion. The converted image 
data is further sent through the signal conversion unit 20 to the display device 21 such as 
25 an electronic viewfinder or a liquid crystal display, whereby an image is reproduced on 
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the display device 21 . 

For the purpose of visual recognition of a subject to be captured, reproduction 
of the subject should be continuously made on the display device 21, and should not be 
limited to a capturing period. Accordingly, data should be continuously sent to the 
5 image display unit 19, thus heavily placing a burden on the bus 14 serving for this data 
transmission. 

In response, in the third preferred embodiment, the local buffers A9 and BIO 
serve to transmit data to the image display unit 1 9 to lighten a burden on a bus band. 

Fig. 4 schematically shows a digital camera comprising an image processing 
10 apparatus 1 according to the third preferred embodiment. In Fig. 4, arrows indicate flow 
of image data captured by the digital camera. Constituent elements which are operative 
in the same manner as those of Figs. 1, 2 and 3 are designated by the same reference 
numerals. 

Image data sent from the CCD sensor 2 or 18 is subjected to the processing at 
1 5 the SPU 6. The processed image data is thereafter sent to the RPU 7 after generation of 
the original image data 16 in the use of the interlace sensor 2, whereas in the use of the 
progressive sensor 1 8, it is directly sent to the RPU 7. The captured image data fi'om the 
RPU 7 is sent through the local buffers A9 and BIO to the image compression and 
expansion unit 13 and to the image display unit 19. The captured image data is 
20 subjected to the processing at the image compression and expansion unit 13 during image 
capturing, whereas in visual recognition of a subject to be captured, it is subjected to the 
processing at the image display unit 19. During visual recognition of a subject to be 
captured, display on the display device 21 is required to be at such a level that allows 
recognition of a subjected to be captured, and does not require data corresponding to 
25 millions of pixels sent from the CCD sensor 2 or 18. Accordingly, image data sent from 
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the RPU 7 during visual recognition is skipped with 240 pixels high by 320 pixels wide, 
for example. The skipped image data is subjected to the sequential process flow as 
discussed in the first preferred embodiment including writing from the RPU 7 and reading 
to the image display unit 19 by means of alternate use of the two local buffers A9 and 
5 BIO under control by the write and read control imits 1 1 and 12. 

Continuity between the RPU 7 and the local buffers A9 and BIO, and continuity 
between the local buffers A9 and BIO, and the image compression and expansion unit 13 
and the image display unit 19, are controlled by purpose-built hardware and a 
purpose-built software program having a switching function of continuity such as the 
10 write and read control units 1 1 and 12. 

The output from the RPU 7 is sent to the local buffers A9 and BIO, and the 
output from the local buffers A9 and BIO is selectively sent either to the image 
compression and expansion unit 13 or to the image display unit 19. That is, the local 
buffers A9 and BIO are selectively available either to the image compression and 
15 expansion unit 13 or to the image display imit 19. More specifically, between the image 
compression and expansion unit 13 and the image display vmit 19, the local buffers A9 
and BIO are allocated to the one placing a heavier burden on the bus 14. 

As discussed, data from the RPU 7 is directly sent through the local buffers A9 
and BIO to the image display unit 19 without data transmission through the bus 14 for 
20 image reproduction as required in the conventional digital camera. A burden placed on a 
bus band is lightened accordingly, to thereby reduce power consumption. The lightened 
burden on a bus band further advantageously leads to enhanced processing speed of the 
image processing apparatus 1 as a whole. 

Fourth Preferred Embodiment 
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As discussed in the description of the background art with reference to Fig. 9, in 
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response to the case in which the CCD sensors 2 and 1 8 have a horizontal line of pixels 
exceeding in length the horizontal line length of the line memory 8 of the RPU 7, the 
original image data 16 is divided into left and right regions. These left and right regions 
are separately processed at the RPU 7 by means of the line memory 8 to generate the 
5 intermediate data 26 on the main memory 5. The intermediate data 26 is subsequently 

subjected to the processing at the image compression and expansion unit 13 to generate 
the compressed image data 17. Such process flow can handle the CCD sensors 2 and 18 
having a horizontal line of pixels exceeding in length the horizontal line length of the line 
memory 8 of the RPU 7, whereas a region for storing the high-volume intermediate data 
10 26 is required in the main memory 5 that contains pixels corresponding in number to 

those of the CCD sensors 2 and 18. In response, in a fourth preferred embodiment of the 
present invention, it will be discussed how the original image data 16 is processed without 
requiring such high-volume intermediate data. 

Fig. 5 schematically shows a digital camera comprising an image processing 
15 apparatus 1 according to the fourth preferred embodiment. In Fig. 5, arrows indicate 
flow of image data captured by the digital camera. Constituent elements which are 
operative in the same manner as those of Figs. 1, 2, 3 and 4 designated by the same 
reference numerals. Fig. 6 is an explanatory view sequentially showing generation of 
data on the main memory 5. With reference to Figs. 5 and 6, the operation of the digital 
20 camera according to the fourth preferred embodiment will be discussed below. 

When the CCD sensors 2 and 1 8 have a horizontal line of pixels exceeding in 
length the horizontal line length of the line memory 8 of the RPU 7, and thus process flow 
cannot sequentially proceed from the SPU 6 to the RPU 7, the original image data 16 is 
generated once in the main memory 5 regardless of whether the CCD sensor 2 or 1 8 is 
used. Next, the original image data 16 is divided and is then subjected to the processing 
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at the RPU 7 by means of the line memory 8. With reference to Fig. 6, the original 
image data 16 is divided into left and right regions in two columns. The left and right 
regions are each further horizontally divided into regions of 8 lines. Hence, assuming 
that the CCD sensors 2 and 18 contain 5000 pixels per horizontal line, for example, data 
5 to be actually processed contains half the number, namely, 2500 pixels per horizontal line. 
As a result, the RPU 7 becomes operative even when the line memory 8 has a capacity of 
as small as 4096 pixels. Regional data P| shown in Fig. 6 after being subjected to the 
processing at the RPU 7 is written as data Qi to a line buffer A22 in the main memory 5. 
Unlike the conventional way which subsequently processes regional data P3, regional data 
10 P2 is subsequently processed and is added as data Q2 to the line buffer A22 on the main 
memory 5. As a result, intermediate data Ri (23) of 8 lines is generated in the main 
memory 5 that has a horizontal line length of pixels corresponding in number to those of 
the horizontal line of the CCD sensors 2 and 18. The image compression and expansion 
unit 1 3 handles data on a block unit basis of 8 pixels high by 8 pixels wide, and hence, is 
15 capable of handling data Ri to generate data S|. These process steps are sequentially 
performed until the compressed image data 17 is generated. 

As discussed, the RPU 7 processes the regional data including Pi, P2, ..., and 
the resultant data R|, R2, ... are subjected to the processing at the image compression and 
expansion unit 13. Even when the CCD sensors 2 and 18 have a horizontal line of pixels 
20 exceeding in length the horizontal line length of the line memory 8, process flow 
continues without waiting for generation of intermediate data of one frame on the main 
memory 5 as required in the conventional digital camera. The fourth preferred 
embodiment further advantageously lowers working buffer capacity in the main memory 
5. Still further, provision of two lines buffers A22 and B23 in the main memory 5, and 
25 alternate use of these buffers A22 and B23 as discussed with reference to the local buffers 
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A9 and BIO in the first and second preferred embodiments realize image processing with 
still higher speed. 

Fifth Preferred Embodiment 

The CCD sensors 2 and 18 may have a horizontal line of pixels exceeding in 
5 length the line memory 8 of the RPU 7 as a result of higher pixel resolution of the CCD 
sensors 2 and 1 8, in which case the fourth preferred embodiment devises reading of image 
data from the original image data 16 to realize image processing. However, the fourth 
preferred embodiment requires a region for storing intermediate data in the main memory 
5, thereby placing a heavier burden on the bus 14 and causing higher power consumption 
1 0 than the first and second preferred embodiments using the local buffers A9 and BIO. A 
fifth preferred embodiment of the present invention as a combination of the second and 
fourth preferred embodiments is responsive to these problems. 

As a circuit configuration of a digital camera, the fifth preferred embodiment 
employs the one shown in Fig. 3 discussed in the second preferred embodiment. As to 
15 image processing, only the reading flow of the original image data 16 discussed in the 
fourth preferred embodiment with reference to Fig. 6 is adopted. 

More specifically, the original image data 16 generated on the main memory 5 
after passing through the CCD sensor 2 or 1 8 and the SPU 6 is read as the regional data Pi 
and P 2 in this order as shown in Fig. 6. The regional data Pi and P 2 thereby read are 
20 subjected to the processing at the RPU 7 by means of the line memory 8. The processed 
data are sent to the local buffers A9 and BIO, not to the line buffer A22 in the main 
memory 5. At this stage, the data R| (23) shown in Fig. 6 is generated in the local 
buffers A9 and BIO. At the time when data transmission is completed, the RPU 7 stops 
reading of the original image data 1 6. Next, the image compression and expeinsion unit 
13 reads the data fi"om the local buffers A9 and BIO and performs compression thereon. 
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The compressed data is thereafter sent as the compressed image data 17 to the main 
memory 5. As this stage, the data Si shown in Fig. 6 is generated as the compressed 
image data 17 on the main memory 5. At the time when data readout from the local 
buffers A9 and BIO is completed, the image compression and expansion unit 13 sends a 
5 control signal notifying completion of data readout. On receipt of this control signal, the 
RPU 7 continues to read the regional data P3 and P4 from the original image data 16, and 
performs image processing thereon. The processed data are sent to the local buffers A9 
and BIO. These process steps are repeatedly performed, whereby only the compressed 
image data 17 is generated on the main memory 5 without generating the intermediate 
10 data 26. 

The fifth preferred embodiment has been discussed as a combination of the 
fourth and second preferred embodiments. As long as the CCD sensors 2 and 1 8 have a 
horizontal line of pixels that does not exceed in length the horizontal line length of the 
local buffers A9 and BIO, the first preferred embodiment may alternatively be combined 
1 5 with the fourth preferred embodiment. Still alternatively, a combination of the third and 
fourth preferred embodiments will be applicable. 

As discussed, the use of the local buffers A9 and BIO of the second preferred 
embodiment combined with the fourth preferred embodiment eliminates the need for 
generating intermediate data on the main memory 5. Such combination further 
20 advantageously lightens a burden to be placed on the bus 14 and reduces power 
consumption. 

While the invention has been shown and described in detail, the foregoing 
description is in all aspects illustrative and not restrictive. It is therefore understood that 
numerous modifications and variations can be devised without departing from the scope 
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of the invention. 




